Perspectival realists often appeal to the methodology of science to secure a realist account of the retention and continued success of scientific claims through the progress of science (e.g., Massimi [2016]). However, in the context of modern physics, the retention and continued success of scientific claims are typically only definable within a mathematical framework. In this article, I argue that this concern leaves the perspectivist open to Cassirer’s (1910/1923) neo-Kantian critique of the applicability of mathematics in the natural sciences. To support this criticism, I present a case study on the conservation of energy in modern physics.